On the mechanical simulation of habit-forming and learning

This paper discusses digital techniques by which habit-forming and learning may be simulated. After classifying the types of simulation mechanisms it discusses types of habit-forming and learning to be simulated, focusing attention upon reinforcement. It uses the language of computer programming to describe the flow of control, and the language of mathematical probability to analyze the effect of various reinforcement functions on the asymptotic behavior of simulating programs. It shows further, again in programming terms, how the “delayed random selector” part of the simulating process may be “factored out” as a separate unit applicable either to habit-forming or learning, which latter are distinguished by whether the reinforcements are applied immediately or upon “comparison with a goal.”Several reinforcement models are considered, including the “linear asymptotic” model used extensively by Bush and Mosteller, two simple “absorbing boundary” models, and a “nonlinear asymptotic” model currently being investigated by Bush, Galanter, and Luce. A sketch is given of the Harris-Bellman-Shapiro analysis of the linear asymptotic model. Contrasted with this, a complete analysis is given of the simpler absorbing boundary model, with explicit proof of eventual absorption, and formulae for probability of absorption in n trials, and the expected number of steps to absorption. Finally, a special example is given of the second absorbing boundary model to show how its structure differs from the others

Adapting to Computer Science

Although I am not an engineer who adapted himself to computer science but a mathematician who did so, I am familiar enough with the development, concepts, and activities of this new discipline to venture an opinion of what must be adapted to in it. Computer and Information Science is known as Informatics on the European continent. It was born as a distinct discipline barely a generation ago. As a fresh young discipline, it is an effervescent mixture of formal theory, empirical applications, and pragmatic design. Mathematics was just such an effervescent mixture in western culture from the renaissance to the middle of the twentieth century. It was then that the dynamic effect of high speed, electronic, general purpose computers accelerated the generalization of the meaning of the word computation This caused the early computer science to recruit not only mathematicians but also philosophers (especially logicians), linguists, psychologists, even economists, as well as physicists, and a variety of engineers. Thus we are, perforce, discussing the changes and adaptations of individuals to disciplines, and especially of people in one discipline to another. As we all know, the very word discipline indicates that there is an initial special effort by an individual to force himself or herself to change. The change involves adaptation of one\u27s perceptions to a special way of viewing certain aspects of the - world, and also one\u27s behavior in order to produce special results. For example we are familiar with the enormous prosthetic devices that physicists have added to their natural sensors and perceptors in order to perceive minute particles and to smash atoms in order to do so (at, we might add, enormous expense, and enormous stretching of computational activity). We are also familiar with the enormously intricate prosthetic devices mathematicians added to their computational effectors, the general symbol manipulators, called computers

Self-Annihilating Sentences: Saul Gorn\u27s Compendium of Rarely Used Clichés

This report was originally issued January 1985. Due to the sudden and untimely loss of Dr. Saul Gorn, the department (CIS) has re-issued this paper for its\u27 1992 series of reports

The Generic Model of Computation

Over the past two decades, Yuri Gurevich and his colleagues have formulated axiomatic foundations for the notion of algorithm, be it classical, interactive, or parallel, and formalized them in the new generic framework of abstract state machines. This approach has recently been extended to suggest a formalization of the notion of effective computation over arbitrary countable domains. The central notions are summarized herein.Comment: In Proceedings DCM 2011, arXiv:1207.682